Do Grammatical Error Correction Models Realize Grammatical Generalization?

There has been an increased interest in data generation approaches to grammatical error correction (GEC) using pseudo data. However, these suffer from several issues that make them inconvenient for real-world deployment including a demand large amounts of training On the other hand, some errors based on rules may not necessarily require amount if GEC models can realize generalization. This study explores what extent generalize knowledge required correcting errors. We introduce analysis method synthetic and real datasets with controlled vocabularies evaluate whether unseen found current standard Transformer-based model fails generalization even simple settings limited vocabulary syntax, suggesting it lacks ability correct provided examples.